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' APPARATUS AND METHOD FOR DETECTING A MOVIN G 
OBJECT IN A SEQUENCE OF COLOR FRAME BI AGES 

Technical Field: 

- The present invention relates to an apparahis and method for detecting 
a moving object in a motion picture , and more spe'cifically , to an apparatus and 
method for detecting a face region from a motion picture by using the color and 
motion information of the face region. 

Background Art: 



Detecting a face region from a motion picmre is one of the prerequisite 
steps in face recognition. Conventional schemes for detecting a face region 
15 have not been used widely, since those schemes are affected by background 
images, and the size and orientation of the face. 

Detection of face regions may be performed by using the information of 
the shape, color, or motion of the faces. 

Using shape information for detecting a face region may be performed 
20 by measuring variation of gray levels and applying the measured values to a 
priori information of the face. However, the scheme should be applied only 
to images containing front side of a face, and the detection result is largely 
affected by background images, and the size and orientation of the face. 

The scheme of using color information suffers from racial deviations of 
25 face colors, because the scheme detects a face region by using the inherent 
color of human faces. Further, the scheme requires large amount of data 
processing, since it uses much more information than the scheme of using shape 
information. However, it becomes more applicable than the scheme of using 
shape information, as hardware technology has recently developed. 
30 The motion information may also be used for detecting face regions in 



- 2 - * 

rndtion pictures wherein an object, i.e. a face, is moving. 
Disclosure of Invention: 

5 It is therefore a principal object of the invention to provide an apparatus 

and method for detecting a moving object wherein the detection is not affected, 
by background images, and the size and orientation of the moving object. 

It is another object of the invention to provide an apparatus and method 
for detecting a moving object mcluding a face region wherein the detection may 

10 be performed faster than the prior art. 

In accordance with one aspect of the present invention to achieve the 
aforementioned object, an apparatus for detecting a moving object in a sequence 
of color frame images is provided. The color frame images have a plurality of 
pixels, each having three color components. The apparatus comprises a color 

15 normalizer for normalizing color components of the color frame image to 
produce a normalized frame image, a color transformer coupled to the coloi 
normalizer for color transforming the normalized frame image to a fnst color 
transformed frame image, the first color transformed frame image having 
intensity levels such that pixels corresponding to said moving object are 

20 emphasized, a frame delay coupled to the color transformer for delaying the 
first color transformed frame image by one frame to produce a second color 
transformed frame image, and a motion detector coupled to the color 
transformer and the frame delay for detecting the motion of the moving object 
and further intensifying the intensity levels of the first color transformed frame 

25 image based on the detected motion. 

Brief Description of Drawings: 



The present invention will become more apparent upon a detailed 
30 description of the best mode for carrying out the invention as rendered below. 



In the description to follow, references will be made to the accompanying 
drawings, where like reference numerals are used to identify like parts in the 
various views and in which; 

Fig. 1 shows a block diagram of the overall system for- detecting a 
5 moving object ' in a color motion picture in accordance with the present 
invention; 

Fig. 2 shows a block diagram of the color' normalizer 100 of Fig. 1; 
Fig. 3a shows a histogram of normalized color components in a typical 
face image. 

.0 Fig. 3b shows a histogram of normalized, color components in a face 

images modeled by using 2-dimensional, Gaussian distribution function.. 

Fig. 4 shows a block diagram of the color transformer 200 of Fig. 1; 

Fig. 5 shows results of color transformation by using GFCD modeling 
of a typical face image; 

l5 Fig. 6 shows a block diagram of the motion detection block 400 of Fig, 

1. - 

. Fig. 7 shows motion detection results of two successive images which 
have a rectangular object. 

Fig. 8 shows motion detection results of two successive- images having 
20 a human, face. 

Best Mode for Carrying out the Invention: - 

Fig. 1 shows a block diagram for illustrating the overall system for 
25 detecting a moving object in a color motion picture in accordance with the 
present invention. The system comprises a color nomializer 100 for 
normalizing color components of a current frame image of the color motion 
picture; a color transformer 200 for color transforming the normalized frame 
image to a color transformed frame image; a frame delay 300 for temporarily 
30 storing the color transformed frame image from die color transformer 200 to 
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delay the color transformed frame image by one frame; and a motion detector 
400 for receiving the color transformed current and previous frame images 
from the color transformer 200 and the frame delay 300 and detecting the 
motion of the moving object. The color transformed frame image, which is 
5 produced by the color transformer, has intensity levels such that pixels 
corresponding to the moving object are emphasized. The motion detector 400 
uses die detected motion of the moving objects to'further mtensify the intensity 
levels of pixels of the moving object. 

Referring now to Fig. 2, the operation of the color normalizer 100 of 

10 Fig. 1 will be explained. 

In color model of RGB color space, for example, any color information 
Q may be represented by three color components, i.e., red, green, and blue 
color components R, G, and B, having different wavelengths. In other words, 
the color mformation Q may be given by Q = (R, G, B). A luminance value 

15 Y may be defined by sum of the three color components. The luminance value 

Y is a measure of a visible radiant energy which produces the sensation of 
brightness. In order to prevent brighmess of an image from affecting the color 
components of an image, the color components of each pixel must be 
normalized by limainance value Y of the pixel; 

20 Fig. 2 shows a block diagram of the color normalizer 100. The color 

normalizer 100 comprises a luminance extractor 130, and two normalizers 110 
and 120. The luminance extractor 130 evaluates a luminance value Y by 
summing red, green, and blue color components R, G, and B for each pixel of 
a frame image, i.e., Y = R + G + B, and outputs it. Then, the normalizer 

25 1 10, for example, red component normalizer receives the Y values and outputs 
normalized color component r by using the luminance value Y and red color 
component R, wherein r is given by (RiY) * 255. Similarly, the normalizer 
120, for example, green component normalizer receives the luminance value 

Y and outputs normalized color component g which is given by g = (R/Y) * 
30 255. Hereinabove, we assumed that the luminance value has depth of 3 bits. 
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and the most bright pixel has the luminance value of 255. By the 
normalization, any color information Q may be represented by two normalized 
color components r and g, i.e. Q = (r, g), since the three normalized color 
components r, g, and b has the relationship of r + g + b = 255. In this way, 

5 the color model of the normalized color space can represents 256-/2 colors, 
while die color model of RGB color space represents 256^ colors. 

Fig. 3a shows histogram of normalized color components in a typical 
face image. In Fig. 3a, two horizontal axis represents two normalized color 
.component values r and g, and the vertical axis represents number of pixels 

10 haviag normalized color components of r and g in a typical face image. 
Distributions of the normalized color components are similar for aU face images 
regardless of backgrounds and brightness. 

Fig. 3b shows histogram of normalized color components ia face image 
which is modeled by 2-dunensional Gaussian distribution function. In this 

1 5 embodiment of the invention, die modeling is performed by using 2-dimensional 
Gaussian distribution as shown ia Fig. 3b. The Gaussian model for color 
distribution La face images is called Generalized Face Color Distribution 
(GFCD). GFCD is defined by mean m and variance E^, in odier words, GFCD 
is represented by GF(m, E^). In diis representation, m = (/r, /g) is the center 

20 point of the 2-dimensional Gaussian distribution, wherein /r and Vg are mean 
values of normalized color components for all pixels ui a face image. 
represents a covariance matrix of normalized color components in a face image. 
In a preferred embodiment, a GFCD for face images may have m = (105, 95) 
and E^ of a, = 20 and = 15. By usiag the GFCD model, an input color 

25 image containing a face may be transformed to a gray-level image where pixels 
having color of the face are emphasized. 

The color transformation using a color distribution model transforms 
normalized color components of pixels to values which represent the proximity 
of die color components to die mean m = (/r, /g) of the Gaussian distribution 

30 in die normalized color space. The color transformation is defined as follows: 
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Z{x, y) =GF{rl,x. y) . g(x. y)) {x.y)£l Eq.1 

.wherein (x, y) is a coordinate of a pixel in the face image, g(x, y) and r(x, y) 
are normalized color components of the pixel at the coordinate (x, y), and GFQ 
is Generalized Face Color Distribution function. The color transformation 
using GFCD produces values which are proportional to the proximity of the 
5 color of the pixel to a human face color. ? 

Referring to Figs. 4 and 5, the operation of the color transformers 200 
of Fig. 1 will be explained. 

Fig. 4 shows a block diagram of the color transformer 200. Color 
transformer 200 comprises blocks 210 and 220. two normalization block 230 

10 and 240. and output block 250. . Block 210 receives the normalized color 
component r from the normalizer 110 and compares it with a, which is 
predetermined in accordance with a particular GFCD model. If the normalized 
color component r is widiin three times of a, around /r, flien. the block 210 
outputs r' = 1 r - /r I . Otherwise, the block 210 outputs a very large value 

15 in order to make Zr = 0. Blocks 230 receives die output value r' and outputs 
transformed value Zr by referring to a look-up table LUT which contains 1- 
dunensional normalized Gaussian number GF(0, O. Similarly, block 220 
receives the normalized color component g and compares it with a^. If the 
normalized color component g is within three times of around /g, then; the 

20 block 220 outputs g' = | g - /g | . Otherwise, the block 220 outputs a very 
large value in order to make Zg = 0. The block 240 receives the output value 
g' and outputs transformed value Zg by referring to a look-up table LUT which 
contains 1 -dimensional normalized Gaussian number GF(0, a^^). Output block 
250 receives Zr and Zg values and evaluates Z = (Zr * Zg) * 255 and outputs 

25 Z. Fig. 5 shows results of color transformation by using GFCD modeling of 
a typical face image. As can be seen in the result image of Fig. 5, areas 
having human face color are emphasized. 

If any background images have color which is similar to the color of 
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human face, the above method may emphasize those background images as well 
as face area. In order to overcome this problem, the present method uses 
motion information of human face. In general, only the human face area is 
moving while background images remain still. Thus, the human face area may 
5 be outstandingly distinguished from background images, if motion mformation 
of the face area is utilized. 

Two methods for extracting motion information from two successive 
frame images are well known in the art. They are region based method and 
feature point based method. Most applications in the art use the region based 

10 method rather than the feature point based method, since the latter generally 
requires post-processing which uses interpolation technique. 

In a typical region based method, the inter-frame differences of intensity 
levels of pixels adjacent a particular pixel are simmied in order to measure the 
motion of the particular pixel. This method is known as Accumulated 

15 Difference Measure (ADM) method. Since the' ADM method uses the 
summation of intensity differences of neighbor pixels, it detects a motion 
robustly against small noisy change of neighbor pixels. However, pixels having 
small differences are ignored by averaging effect, and the measurements are 
largely affected by the determination of threshold value. 

20 Thus, the present invention preferably performs motion detection by 

counting pixels whose inter-frame differences are larger than a threshold value 
in a window having predetermined size. This motion detection measure will 
be called Unmatching Pixel Count (UPC) measure and may be written as 
follows: 

UPCU,y.t)=J2 E UU,j,t:) Eg. 2 



25 , ^ ^(^^ I O-Z(iJj-l) I >Th 

otherwise 



[O, otherw. 

wherein Th is a threshold value for determining whether the intensity values of 
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pixels in two successive frame images are matched. In the UPC method, since 
matching is tested in a pixel by pixel manner and the results are accumulated, 
the averaging effect is reduced such that overlapping area causing a small 
intensity change is clearly captured. 

5 In order to detect motion of a color image, die color information 

explained hereinabove may be used. It may be recognized by those skilled in 
the art that the intensity level of each pixel in the color transformed imaee 
which was transformed by using the color transformation method of GFCD 
modelling, represents a probability fliat die pixel is in a face area. The color 

10 information of die color transformed image may be incorporated in Eq. 2 by 
weighting the counts with the intensity level of the pixel of the color 
transformed image. This new motion detection measure will be referred to as 
Weighted Umnatching Pixel Count (WUPC). The WUPC(x, y, t) is evaluated 
by performing fuzzy- AND operation between die UPC measure UPC(x, .y, t) 

15 and the intensity level of die color transformed image Z(x, y, t). The WUPC 
measure is given by following equation: 



!;t - i=X*N j=y*N 



WUPC{x,y, t) ^Z{x,y, t) ^ ^ U(i,j.t) Eg. 3 

rrr ■ rj = 'f\^(''j' t)-Z(i,j,t-l)\ >Th 
y 0, otherwise 

20 wherein the operator ® is fiizzy-AND operator. This WUPC measure 
emphasizes motion of the pixels having color which is similar to color of a 
human face, while deemphasizing motion of the odier pixels, i.e., pixels of 
background images. 

In Eq. 3, die threshold value Th may be obtained adaptively by using die 

25 transformed image value Z(x, y, t) for each pixel. In odier words, die Th 
becomes small for pixels which are in a face area and large for odier pixels. 
In diis way, die WUPC measure is more sensitive to motion of pixels in a face 
area dian motion of die other pixels, such diat even small motion of face area 
may be detected. In a preferred embodiment of the present invention, die 

30 direshold Th may be obtained by using Sigmoid function as follows: 

Slope of Sigmoid function becomes steep as Q is lowered. Thus, Sigmoid 



IJl 



Th(Z) = 25L_ 

Z{x. y, C) -2S5/2 

1 + e ° 
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function is a lot like a step-function shape for small Q. It is appropriate to use 
Sigmoid function as a function for deterniinijig threshold Th of Eq. 3, since the 
Sigmoid function outputs small threshold Th for a large input value and large 
Th for small input value. Further, the Sigmoid function is nonlinear m that the 
5 function does not respond abruptly to maximimi or minimum input values. 
This method for adaptively counting. unmatched pixels will be called Adaptive 
Weighted Unmatching Pixel Count (AWUPC) measure. 

Fig. 6 shows a block diagram of the motion detection block 400 of Fig. 
1. The motion detection block 400 comprises line buffers 410 and 420 for 

10 temporarily storing color transformed images and outputting intensity levels of 
pixels in a window of a predetermined size; an adaptive threshold value 
generator 440 for generating threshold th in response to intensity level of a 
central pixel in the window; an unmatching pixel value generator 430 for 
receiving the threshold Th and present and previous frames of the color 

15 transformed image from line buffers 410 and 420 to determine whether the 
pixels in the window of the present frame match with the pixels in the window 
of the previous frame by evaluating unmatching pixel value U(x, y, t) of Eq. 
3; an unmatching pixel counter 450 for counting the pixels which are 
determined to be unmatching pixels in the unmatching pixel generator 430; and 

20 a fuzzy-AND operator 460 for performing fuzzy- AND operation on the outputs 
of the unmatching pixel counter 450 with the intensity level Z(x, y, t) of the 
pixel of the color transformed image from the line buffer 420. The line buffer 
420 outputs intensity level Z(i, j, t) in a window of the present frame, while the 
line buffer 410 outputs intensity level Z(i, j, t-1) in a wmdow of the previous 

25 frame of the color transformed image. The window size is (2N + 1) x (2N + 
1) such that the line buffers 410 and 420 output intensity levels of (2N -I- 1) x 
(2N + 1) pixels. The threshold value generator 440 receives the intensity level 
of the central pixel Z(x, y, t) among the window of the present frame and 
evaluates a threshold value Th according to Eq. 4. The unmatching pixel 

30 generator 430 receives outputs from the line buffers 410 and threshold value 
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Th from the threshold value generator 440. Then, the unmatching pixel 
generator 430 evaluates the u(i, j. t) of Eq. 3 for a central pixel Z(x, y, t) of 
the window. The unmatching pixel counter 450 receives the outputs of the 
unmatching pixel generator 430 and counts the number of unmatching pixels. 
5 The fuzzy-AND operator 460 outputs a color motion information by performing 
a fuzzy-AND operation between the intensity level of the central pixel Z(x, y, 
t) from the line buffer 420 and the outputs from the unmatching pixel counter 
450. 

ReferriQg to Figs. 7 and 8, motion detection results of the present 

10 invention will be compared with those of the prior art." Fig. 7 shows motion 
detection results of two successive images which have a rectangular object. 
Fig. 7a shows the first images having a rectangular object with background 
images. Pixels in the rectangular object of Fig. 7a have random gray levels of 
200 - 230, while the gray ' levels of pixels in the background images are random 

15 values of 0 - 30. Fig. 7b shows the .second image wherein the rectangular 
object is moved by 50 pixels in x and y directions. Fig. 7c shows the motion 
detection results according to the conventional ADM method. The areas, 
whose inter-frame differences of intensity levels are large, are emphasized. 
However, those areas where the rectangular objects of the two subsequent 

20 images are overlapped are not emphasized. Thus, the motion is not well 
detected by the conventional ADM method when the two objects of sebsequent 
frmes are overlapped. Fig. 7d shows a motion detection result according to the 
UPC measure. In Fig. 7d, those areas where the rectangular objects of the two: 
subsequent images are overlapped are also emphasized. The UPC measure can 

25 detect motion of a moving object better than the ADM method, since it is able 
to detect overlapping areas of a moving object. Fig. 7e shows motion detection 
result according to the WUPC measure. The areas of the moving object having 
a desired color in die first image are emphasized. Fig. 7f shows the motion 
detection results by usmg AWUPC measure according to die present mvention. 

30 Fig. 8. shows motion detection results of two successive images having a human 
face. Figs. 8a shows GFCD color transformed image of the first images 
having a human face with background images. Fig. 8b shows GFCD color 
transformed image of the second image wherein the human face is moved 
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slightly. Fig. 8c shows the motion detection results according to the 
conventional ADM method. The motion is not well detected by the 
conventional ADM method as shown in Fig. 8c. Fig. 8d shows a motion 
detection result according to the UPC measure. Figs. 8e and 8f respectively 
5 show motion detection result according to the WUPC measure and AWUPC 
measure in accordance widi the present invention. 



